THE GROUP STRUCTURE OF THE NORMALIZER OF r (iV) 



FRANCESC BARS 



Abstract. We determine the group structure of the normalizer of Fo(N) in 
5L2(K) modulo ro(JV). These results correct the Atkin-Lehner statement [1, 
Theorem 8] . 

1. Introduction 

The modular curves Xq(N) contain deep arithmetical information. These curves 
are the Riemann surfaces obtained by completting with the cusps the upper half 
plane modulo the modular subgroup 

T (N) = {( * c b d ) eSL 2 (Z)\ceZ}. 

It is clear that the elements in the normalizer of To(N) in 5X2 (M) induce auto- 
morphisms of Xq(N) and moreover one obtains in that way all automorphisms of 
X (N) for N 7^ 37 and 63 [3]. This is one reason coming from the modular world 
that shows the interest in computing the group structure of this normalizer modulo 
To(N). 

Morris Newman obtains a result for this normalizer in terms of matrices [5] , [6] , 
see also the work of Atkin-Lehner and Newman [4]. Moreover, Atkin-Lehner state 
without proof the group structure of this normalizer modulo Tq(N) [1, Theorem 
8] . In this paper we correct this statement and we obtain the right structure of the 
normalizer modulo To(N). The results are a generalization of some results obtained 
in [2]. 

2. The Normalizer of T (N) in SL 2 (R) 

Denote by Norm(r (iV) the normalizer of T Q (N) in S*L 2 (R). 

Theorem 1 (Newman). Let N — a 2 q with o~,q € N and q square-free. Let e be the 

gcd of all integers of the form a — d where a, d are integers such that f ^ d ) ^ 

T (N). Denote by v := v(N) := gcd(cr,e). Then M g Norm(L (A r )) if and only if 
M is of the form 

with r,u, s,l € Z and 5\q, A|^. Moreover v = 2^3 W with /1 = min(3,[hv2(N)]) and 
w = mm(l, [\v^{N)\) where v Pi (N) is the valuation at the prime Pi of the integer 
N. 
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This theorem is proved by Morris Newman in [5] [6], see also [2, p. 12-14]. 
Observe that if gcd(5A,6) = 1 we have gcd(<5A 2 , jgs) — 1 because the determi- 
nant is one . 

3. The group structure of Norm(ro(AO)/r (AO 

In this section we obtain some partial results on the group structure of Norm(To(-/V)). 
Let us first introduce some particular elements of SX 2 (R). 

Definition 1. Let N be fixed. For every divisor to' of N with gcd(m', N/m!) = 1 
the Atkin-Lehner involution w m > is defined as follows, 

1 / m'a b \ rlT 
W -' = ^\Nc m'd ) 6 SL ^ 

with a, b,c,d e Z. 

Denote by S v * = ^ * ^ ) with v> e N \ Atkin-Lehner claimed in [1] the 
following: 

Claim 2 (Atkin-Lehner). [1, Theorem 8] The quotient Narm.(T (N))/T (N) is the 
direct product of the following groups: 

(1) {w q v q (N)} for every prime q, q > 5 q | N. 

(2) (a) Ifv 3 (N) = 0, {1} 

(b) Ifv 3 (N) = l, {w 3 } 

(c) Ifv 3 (N) = 2, {w 9 ,S 3 }; satisfying w\ = S$ = (w 9 5 3 ) 3 = 1 f/ac£or of 
order i2j 

(d) Ifv 3 (N) > 3; {w 3l , 3i N),S 3 }; where w 3 2 „ 3(N) = S| = 1 andtOgvawSsuVsw 
commute with S 3 (factor group with 18 elements) 

(3) Let be A = v 2 (N) and /i = min(3, [-|]) and denote by v" = 2 M i/ie we have: 

(a) J/A = :{1} 

(b) //A=1;{ W2 } 

(c) //A = 2/x; {u> 2 „ 2 (jv), mi/i rte relations w^ V2(N) — S%„ — (w 2 v 2 (n)S v ") 3 = 
1, where they have orders 6,24, an d 96 for v — 2,4,8 respectively. 

( One needs to warn that for v = 8 the relations do not define totally 
this factor group). 

(d) If X > 2fi; { w 2V2 (N) , S v »} ; w^ 2(N) = S%" = 1. Moreover, S v » com- 
mutes with w 2 v 2 (n) S v "W 2 v 2 (.n) (factor group of order 2v" 2 ). 

Let us give some partial results first. 

Proposition 3. Suppose that v(N) = 1 (thus 4 { N and 9 \ N). Then the Atkin- 
Lehner involutions generate Norm(r ( N)/T (N) and the group structure is 

7r(iV) 

f[ Z/2Z 

i=l 

where ir(N) is the number of prime numbers < N. 

Proof. This is classically known. We recall only that w mm i — w m w m i for (to, to') = 
I and easily w m w m > — w m >w m ; the the result follows by a straightforward compu- 
tation from Theorem 1, see also [2, p. 14]. □ 
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When v(N) > 1 it is clear that some element S v > appears in the group structure 
of Norm(r (^))/r (A r ) from Theorem 1. 

Lemma 4. If 4\N the involution S 2 € Norm(ro(A^)) commutes with the Atkin- 
Lehner involutions w m with gcd(m, 2) = 1 and with the other S v > . 

Proof. By the hypothesis the following matrix belongs to To(N) 

( 2mk 2 +2Nt+mkNt (2+2m)(2m+2mk+Nt) \ 



W m S 2 W m S 2 - I Nt(2m+2Lk+Nt) , jy. , m", km , Aft 2 

\ 2m m 2 4m 



1 / A 2 5r 

u = 



following the notation of Theorem 1. Then: 



□ 



Proposition 5. Let N = 2 V ^ N ^ WiV™* > with pi different odd primes and assume 
that V2(N) < 3, v 3 (N) < 1. Then Atkin-Lehner's Claim 2 is true. 

For the proof we need two lemmas. 

Lemma 6. Let u e Norm(r (iV)) and write it as: 

,2; 




wahu= s > n 2 . , if gcd(5,2) = l, 



1 / 2r" — \ 

™A 2 f" = ^ 2v" j ' lf gcd(<5 ' 2) = 2 - 

Proof. This is an easy calculation. □ 
We study now the different elements of the type 




1 

= V2 

Observe that &(,,,) only appears when N = 0(mod 8). 

Lemma 7. For N = A(mod 8) all the elements of the normalizer of type a(r' , u', s', v') 
belong to the order six group {S 2l W4\S 2 — w\ — (W4S2) 3 = 1}. 

Proof. Straightforward from the equalities: 

a(r', it', s', v') e T (N) & s' = u' = 0(mod 2) 
a{r',u',s',v')S 2 S T (iV) & r' = v' = u' = 1 s' = 0(mod 2) 
a(r', u, s', v')wi G r (iV) s/ed'eOu'e/e l(mod 2) 
a(r', u', s', w')^^ € r (JV) <s> / = u' = s' = 1 t;' = 0(morf 2) 
a(r', it', s', w')5 2 w 4 e T (iV) 1/ = u' = s' = 1 r' = 0{mod 2) 
a(r', it', s', v')S 2 w 4 S 2 € T (iV) Or' ed' = ,s'e 1 ti' e 0(mod 2) 

□ 
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Lemma 8. Let N be a positive integer with V2(N) = 3. Then all the elements 
of the form a(r',u',s',v') and b(r" ,u" , s" ,v") correspond to some element of the 
following group of 8 elements 

{S 2 , W 8 \S 2 =wl = l, S 2 W 8 S 2 W 8 = W 8 S 2 W 8 S 2 } 

Proof. If follows from the equalities: 

a(r',u', s', v') G T (N) r' = v' = 1, u' = s' = 0(mod 2) 
a(r' ,u, s', v )S , 2 G Tq(N) r = v = u = 1, s = 0(mod 2) 
a(r' , u, s', v')w 8 S2W 8 G Tq(N) r = v = s = 1, u = Q(mod 2) 
a(r', u', s', v')S 2 w s S2W$ G r (iV) & r' = v' = s' = v' = \{mod 2) 
b(r", u", s", v")w s G T Q (N) & r" = v" = 0, u" = s" = l(mod 2) 
b(r", u", s", v")S 2 w 8 S2 G r (A^) & r" = v" = u" = s" = \{mod 2) 
b(r", u", s", v")S 2 w 8 G T (iV) r" = 0, u" = s" = v" = \{mod 2) 
b(r", u", s", v")w 8 S 2 G r (JV) v" = 0, u" = s" = r" = l(mod 2) 

□ 

We can now proof Proposition 5] . 

Proof. [ of Proposition 5] Let N = J], P? , with Pl different primes and 

assume that 9 \ N. If V2{N) < 1 we are done by proposition 3. Suppose v 2 {N) = 2 
and let u G Norm(r (iV)). By lemmas 6 and 7, w$u — a, a <E {S2,W4\S 2 —w\ — 
(w45 , 2) 3 = 1 and it follows that 11 — wsa. Since w$ (((5,2) = 1) commutes with 

52 and the Atkin-Lehner involutions commute one to each other, we are already 
done. In the situation 8||iV the proof is exactly the same but using lemmas 6 and 
8 instead. □ 

4. Counterexamples to Claim 2. 

In the above section we have seen that Atkin-Lehner's claim is true if v(N) < 2 
i.e. for V2(N) < 3 and Vs(N) < 1. Now we obtain counterexamples when V2(N) 
and/or Vs(N) are bigger. 

Lemma 9. Claim 2 for N — 48 is wrong. 

Proof. We know by Ogg [7] that A (48) is an hyperelliptic modular curve with 
hyperelliptic involution not of Atkin-Lchncr type. The hyperelliptic involution 
always belongs to the center of the automorphism group. We know by [3] that 
Aut(A (48)) = Norm(r (48))/r (7V). Now if Claim 2 where true this group would 
be isomorphic to Z/2 x II4 where II n is the permutation group of n elements. It 
is clear that the center of this group is Z/2 x {1}, generated by the Atkin-Lehner 
involution w 3 , but this involution is not the hyperelliptic one. □ 

The problem of N = 48 is that S4 does not commute with the Atkin-Lehner 
involution W3; thus the direct product decomposition of Claim 2 is not possible. 
This problem appears also for powers of 3 one can prove, 

Lemma 10. Let N = 3 V3< - N ^ UiPi' where 

Pi are different primes o/Q. Impose that 

53 G Norm(r (-/V)). Then S3 commutes with w p ^i if and only ifp™ 1 = l(modulo 3). 
Therefore if some p™* = —l(modulo 3) the Claim 2 is not true. 
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Proof. Let us show that 5*3 does not commute with w p ™i if and only if p" 1 = — l(mod 

( p n 'k 1 \ 
3). Observe the equality w = }„. * „, : 

W p »iS3W p niSl = 

J_/ (p^+^l + EiJ*) p r .fc(^ + i) + (25*+i)(2^+^) \ 

For this element to belong to To(N) one needs to impose — — h ^3— G Z. Since 
p" s = 1 o — I (mod 3) it is needed that k = l(mod 3). Now from det(io Pi ) = 1 we 
obtain that p^k = l(mod 3); therefore = \(mod 3). □ 

5. The group structure of Norm(r (A r ))/r (A r ) revisited. 

In this section we correct Claim 2. We prove here that the quotient 

Norm(r (iv))/ro(AO 

is the product of some groups associated every one of them to the primes which 
divide N. See for the explicit result theorem 16. 

Theorem 11. Any element w € Norm(r (iV)) has an expression of the form 

u> = u> m Q, 

where w m is an Atkin-Lehner involution of Tq(N) with (m,6) = 1 and be- 
longs to the subgroup generated by S v t^ and the Atkin Lehner involutions w 2 v 2 (n), 
w 3 v 3 (N). Moreover for gcd(v(7V), 2 3 ) < 2 the group structure for the subgroup 
< S V2 ( v (N)),w 2 v2m > and < S V3 ( v ( N ^,w 3 v 3 (n) > of < S v ( N ),w 2 v 2 (n),w 3 v 3 (n) > 
is the predicted by Atkin-Lehner at Claim 2, but these two subgroups do not neces- 
sary commute withe each other element-wise. 

Proof. Let us take any element w of the Norm(r (-/V)). By Theorem 1 we can 
express w as follows, 

w -^( rA ^ \_ J_( r5A* * 

Let us denote by U = 2 V2 ^3 V ^ N \ Write A' = gcd(A, N/U) and S' = gcd(6,N/U); 
then we obtain 

Observe that if v(N) = 1 we already finish and we reobtain proposition 3. This is 
clear if gcd(7V, 6) = 1; if not, the matrix is the Atkin-Lehner involution at 

(£) 2 JUn. 

Now we need only to check that any matrix of the form 

m o _ 1 ( r 'y^ 2 ^v) ] 

y 1 ) A /T7T7 Nt' „/i/A\2 
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is generated by SWjv) and the Atkin-Lehncr involutions at 2 and 3 which are the 
factors of jt(-^t) 2 - To check this observe that Q = fl 2 ^3 with 



r » 2 "2(^(fr) 2 ) 



We only consider the case for £l 2 , the case for the is similar. We can assume 
that 2 V2 ( atV 5 / 5 ') = 1 substituting Q 2 by w 2 v 2 (n)Q 2 if necessary. Thus, we are 

(r' u ' \ 

Nt , 2"2(«>(N)) I _ J^OW f Qr g 0me ^ W g 

2"2(«<«» V J 

( r' v! \ — 
can obtain S^ V2iv(N)) il 2 = I jvt' / I i name this matrix by ^2- Then, it is 

easy to check that w 2V2 (n) S I 2V2{vlNy) w 2 v 2 (N)fl 2 € ro(^V) for some i. 

Similar argument as above are obtained if we multiply w by w m on the right, 
i.e. ww m is also some as above obtaining similar conclusion. 

Let us see now that the group generated by S^^tv)) an d the Atkin-Lehner invo- 
lutions at 2, and the group generated by S V3 ( v (n)) and the Atkin-Lehner involution 
at 3 have the structure predicted in Claim 2 when gcd(v(7V), 2 3 ) < 2. We only 
need to check when v(N) is a power of 2 or 3 by (2). For v(N) = 1 the matrix 
(1) is w s , a ^ 2 (we denote wi := id) (we have in this case a much deeper result, 

J 7 * A 7 ' 

see proposition 3). Take now v(N) =2. If I = gcd(3, 6/5') let Q = wifl'; the 
matrix O' is as (1) but with gcd(3, 5/5') — 1, and jt-^fz is only a power of 2. 
Then ft' e< S2,w 2 v 2 (n) >, let us to precise the group structure. For v(N) = 2 
we have v 2 (N) = 2 or 3, and we have already proved the group structure of Claim 
[1] in lemmas 7,8 (we have moreover that Claim 2 is true because S 2 commutes 
with the Atkin-Lehner involutions w p ^i if {pi,2) = 1, see proposition 5). Assume 
now v(N) =3. If I = gcd{2,5/5') and £1 = Wl fl' then n' is as (1) but with 
gcd(2, 5/5') = 1, and jr-^n is only a power of 3. Then Q! e< S , 3,w 3 v 3 (iv) >, let us 
to precise the group structure. For v(N) = 3 we have v 3 (N) > 2. Let us begin with 
V3(N) = 2, then fl' is of the form 

n'= ( ^ J ) =:a(r>',i>') 

(from the formulation of Theorem 1 we can consider ^7 = 1 = ^ because the 
factors outside 3 does not appear if we multiply for a convenient Atkin-Lehner 
involution, and for 3 observe that under our condition A = 1) and we have 

a{r', u', t', v') <E r (A) ^ t' = u' = 0(mod 3) 

a(r',u',t',v')w 9 e r (N) r = v = 0(mod 3) 
a{r', u', t', v')S 3 € r (A) <S> r' + u' = t' = 0(mod 3) 
a(r', u', t', v')S% € T (A) 2r' + u' = t' = 0(mod 3) 
a(r',u', t', v')S 3 w 9 G T (N) <=> r' = qt' + v' = 0(mod 3) 
air', u', t', v')S 2 w 9 € T (N) & r' = 2qt' + v' = 0{mod 3) 
a(r', u', t', v')w 9 Sj £ T (N) <=> r' + u' = v' = 0{mod 3) 
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a(r', u', t', v')w 9 S 3 € r (iV) & r' + 2u' = v' = 0(mod 3) 

a(r', u', t', v')w 9 S%w 9 e T (N) <=> u' = qt' + v' = 0(mod 3) 

a{r', u\ t\ v')S$w 9 Si e T (N) u' = 2qt' + v' = 0(mod 3) 

a{r', u\ t\ v')Slw 9 S 3 S r (AT) & r' + u' = 2t'q + v' = 0(mod 3) 

a{r', u', t', v')S 3 w 9 Sl e T (iV) 2r' + u' = gi' +v' = 0(mod 3) 

and these are all the possibilities, proving that the group is {S3, W9IS3 = w 9 — 
(w 9 S 3 ) 3 — 1} of order 12. Observe that S3 does not commute with w 2 (see for 
example lemma 7). 

Suppose now that v 3 (N) > 3. We distinguish the cases v${N) odd and v 3 (N) 
even. Suppose v 3 (N) is even, then ^ = 1 and O' has the following form 

Nt' 

A' \ 3 17 V A' 

with a := A/A' dividing S^W/ 2 ! -1 . Since this last matrix has determinant 1 
we see that a satisfies gcd(a, N/(3 2 a 2 )) = 1; thus a = 1 or a = S^W/ 2 ! -1 . 
( r' — \ 

Write a(r', u' , t' , v ') = Nt , 3 , when we take a = 1 and b(r ,u,t ,v) = 
\ — v J 

n, ] fJ ,,S)/S-n when a = 31-W/21-1. It is easy to check 

that b(r' ,u' ,t' ,v') — w 3 v 3 (N)a(r' , u', t', v') and that the group structure is the pre- 
dicted in a similar way as the one done above for v(N) — 2. Suppose now that 
v 3 (N) is odd, then Jr is 1 or 3 and A divides 3MJV)/2]-i_ Now f rom det() = 1 
we obtain that the only possibilities are ^ = 1 = ^ name the matrices for this 
case following equation 1 by a(r' ,u' ,t' ,v'), and the other possibility is j, = 3 
and -^t = sl" 3 ^/ 2 ! -1 , write the matrices for this case following equation 1 by 
c(r',u',t',v'). It is also easy to check that c(r',u',t',v') — w 3 v 3 (.N)d(r" , u", t", v"), 
and that the group structure is the predicted. 

□ 

Corollary 12. Let N = 3 V3 ^ fliP?* > viithpt different primes such that gcd(pi, 6) = 
1. Suppose that v(N) = 3 and p™ 1 = l(mod 3) for all i. Then Claim 2 is true. 

Proof. From the proof of the above theorem 11 for v(N) = 3 with v 3 (N) > 2, lemma 
10, and that the general observation that the Atkin-Lehner involutions commute 
one with each other we obtain that the direct product decomposition of Claim 2 is 
true obtaining the result. □ 

Now we shows the corrections to Claim 2 for v(N) = 4 and v(N) — 8, about the 
group structure of the subgroup of Norm(r (A^))/r (iV) generated for S 2 k and the 
Atkin-Lehner involution at prime 2. 

Proposition 13. Suppose v(N) = 4, observe that in this situation v 2 (N) = 4, or 5. 
Then the group structure of the subgroup < w 2 v 2 (n) , S4 > of Norm(ro(7V))/r (A r ) 
is given by the relations: 

(1) For v 2 {N) = 4 we have Sj = w 2 w = {w w S 4 ) 3 = 1. 

(2) For v 2 {N) = 5 we have Sj = w% 2 = {w 32 S 4 ) i = 1. 
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Proof. It is a straightforward computation. Observe that for V2{N) = 4 the state- 
ment coincides with Claim 2 but not for V2(N) = 5, where one checks that S4 does 
not commute with 1032841032- D 

Proposition 14. Suppose v(N) — 8 and V2(N) even (this is the case (3)(c) in 
Claim 2). Then the group < w 2 v 2 (n> , Ss >C Norm(To(-/V))/To(iV) satisfies the 
following relations: 5§ = w^ 2(N) = 1, and 

(1) for v 2 (N) — 6 we have (w^Ss) 3 = 1, 

(2) for v 2 (N) > 8 we do not have the relation (w 2 v 2 in) Ss) 3 — 1, 

(3) for V2(N) > 10 we have the relation: Ss commutes with w 2 v 2 ( N )Ssw 2 v 2 {n) , 

(4) for V2(N) = 6 or 8 we do not have the relation: Ss commutes with the 
element w 2 v 2 (n)S$w 2 -u 2 (n) . 

(5) For v 2 (N) = 8 we have the relation: w 2 r,aSsw 2 5eSsw 2 5QSsW 2 5QSs = 1. 

Proof. Straightforward. □ 

Proposition 15. Suppose v(N) = 8 andv2(N) odd (this is the case (S)(d) in Claim 
2). Then the group < w 2 v 2 (n),Ss >C Norm(r (-/V))/r (A r ) satisfies the following 
relations: 5f = w^ V2{N) — 1, and 

(1) }orv 2 {N) = 7 (w 12S S s ) A = I, 

(2) for v 2 (N) > 9 we do not have the relation (w 2 v 2 (n)Ss) 4 — 1, 

(3) for v 2 (N) > 9 we have the Atkin-Lehner relation: Ss commutes with 

W 2 v 2 (N) SsW 2 v 2 (N) , 

(4) for v 2 (N) = 7 we do not have that Ss commutes with wi 2 sSsWi 2 s- 
Proof. Straightforward. □ 

Let us finally write the revisited results concerning Claim 2 that we prove; 

Theorem 16. The quotient Norm(T (N))/T (N) is a product of the following 
groups: 

(1) {w qVg (_N)} for every prime q, q > 5 q | N. 

(2) (a) Ifv 3 (N) = 0, {1} 

(b) Ifv 3 (N) = l, {w 3 } 

(c) Ifv 3 (N) = 2, {w 9 ,S 3 }; satisfying w$ = 5f = {w 9 S 3 ) 3 = 1 (factor of 
order 12) 

(d) Ifv 3 (N) > 3; {w 3l , 3i N), S 3 }; where iu^„ 3(JV) = 5f = 1 and w 3 v 3 (n) S 3 w 3 v 3 (n) 
commute with S 3 (factor group with 18 elements) 

(3) Let be A = V2(N) and \i = min(3, [-|]) and denote by v" = 2 M the we have: 

(a) //A = 0;{l} 

(b) If X = l;{w 2 } 

(c) If X — 2[i and 2 < A < 6; {w 2 v 2 (.n) , S v »} with the relations w^ 2(N) — 

S^„ — (w 2l , 2 (N)S v ") 3 — 1, where they have orders 6,24, an d 96 for 
v = 2, 4, 8 respectively. 

(d) // A > 2 M and 2 < A < 7; { w 2V2m , S v »}; u£, 2( „, - S#,' = 1. 

Moreover, (w 2 v 2 (N) Sv" ) — 1 . 

(c),(ci) If A > 9; {w 2 i. 2 (n) , 5s} u>z£/i i/ie relations w^ V2(N) = 5§ = 1 and 58 
commutes with w 2 v 2 (n) SsW 2 v 2 (n) . 
(c) If X = 8; {w 2 „ 2 (w) , 5s} wit/i i/ie relations w^ 2(N) = 5| = 1 and 

W2565 8 ^256 58 W256 5|^2565| = 1. 
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Observation 17. One needs to warn that for the situation v(N) = 8 possible the 
relations does not define totally the factor group, but it is a computation more. 

Observation 18. The product between the different groups appearing in theorem 
16 is easily computable. Effectively, we know that the Atkin-Lehner involutions 
commute, and S^W)) commutes with S 3 v 3 (v(n)) . Moreover S2 commutes with 
any element from lemma 4- Consider w p n an Atkin-Lehner involution for Xo(N) 
with p a prime. One obtains the following results by using the same arguments 
appearing in the proof of lemma 10; 

(1) let p be coprime with 3 and 3\v(N). S3 commutes with w p n if and only if 
p n = l{modulo 3). If p n = —l(modulo 3) then Wp^S^ = S^uip** . 

(2) Let p be coprime with 2 and 4\v(N). S4 commutes with w p n if and only if 
p n = \{modulo 4). If p n = —l(modulo 4) then w p nSi — S\w p n. 

(3) Let p be coprime with 2 and 8|w(A r ). Then, w p nS% — S%w p n if p n = 
k(modulo 8), in particular Sg commutes with w p n if and only if p n = 
l(modulo 8). 
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